将差异化随机梯度下降（DPSGD）应用于培训现代大规模神经网络（例如基于变压器的模型）是一项艰巨的任务，因为在每个迭代尺度上添加了噪声的幅度，都具有模型维度，从而阻碍了学习能力显著地。我们提出了一个统一的框架，即$ \ textsf {lsg} $，该框架充分利用了神经网络的低级别和稀疏结构，以减少梯度更新的维度，从而减轻DPSGD的负面影响。首先使用一对低级矩阵近似梯度更新。然后，一种新颖的策略用于稀疏梯度，从而导致低维，较少的嘈杂更新，这些更新尚未保留神经网络的性能。关于自然语言处理和计算机视觉任务的经验评估表明，我们的方法的表现优于其他最先进的基线。

我们在差异隐私（DP）的洗牌模型中研究高斯机制。特别是，我们表征了该机制的r \'enyi差异隐私（RDP），表明它是形式：$$ \ epsilon（\ lambda）\ leq \ leq \ frac {1} {\ lambda-rambda-1} \ log \ left（ \ frac { } \ binom {\ lambda！} {k_1，\ dotsc，k_n} e^{\ sum_ {\ sum_ {i = 1}^nk_i^2/2 \ sigma^2} \ right）由高斯RDP限制在上面，而不会改组。混乱的高斯RDP在组成多种DP机制方面是有利的，在该机制中，我们证明了其对散装模型的隐私保证的最新近似DP组成定理的改进。此外，我们将研究扩展到了次采样的洗牌机制和最近提出的洗牌机制，这些机制是针对分布式/联合学习的协议。最后，对这些机制进行了一项实证研究，以证明在分布式学习框架下采用洗牌高斯机制来保证严格的用户隐私的功效。

最近对具有正式隐私保证的分布式计算的研究，例如联合学习的差异私有（DP），利用每回合中客户的随机抽样（通过亚采样进行的隐私放大）来达到令人满意的隐私水平。然而，实现这一目标需要强大的假设，这些假设可能无法实践，包括对客户的精确和统一的亚采样，以及高度信任的聚合器来处理客户的数据。在本文中，我们探讨了一个更实用的协议，改组了办理登机手续，以解决上述问题。该协议依靠客户端做出独立和随机的决定来参与计算，释放服务器发射的亚采样要求，并启用客户端辍学的强大建模。此外，采用了称为洗牌模型的较弱的信任模型，而不是使用受信任的聚合器。为此，我们介绍了新工具来表征洗牌的r \'enyi差异隐私（RDP）。我们表明，我们的新技术在隐私保证中至少提高了三次，而在各种参数制度下使用近似DP的强大组成的人进行了三倍。此外，我们提供了一种数值方法来跟踪通用洗牌机构的隐私，包括具有高斯机制的分布式随机梯度下降（SGD）。据我们所知，这也是文献中分布式设置下本地/洗牌模型中高斯机制的首次评估，这可能具有独立的兴趣。

在2015年和2019年之间，地平线的成员2020年资助的创新培训网络名为“Amva4newphysics”，研究了高能量物理问题的先进多变量分析方法和统计学习工具的定制和应用，并开发了完全新的。其中许多方法已成功地用于提高Cern大型Hadron撞机的地图集和CMS实验所执行的数据分析的敏感性;其他几个人，仍然在测试阶段，承诺进一步提高基本物理参数测量的精确度以及新现象的搜索范围。在本文中，在研究和开发的那些中，最相关的新工具以及对其性能的评估。

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.

Creating compelling captions for data visualizations has been a longstanding challenge. Visualization researchers are typically untrained in journalistic reporting and hence the captions that are placed below data visualizations tend to be not overly engaging and rather just stick to basic observations about the data. In this work we explore the opportunities offered by the newly emerging crop of large language models (LLM) which use sophisticated deep learning technology to produce human-like prose. We ask, can these powerful software devices be purposed to produce engaging captions for generic data visualizations like a scatterplot. It turns out that the key challenge lies in designing the most effective prompt for the LLM, a task called prompt engineering. We report on first experiments using the popular LLM GPT-3 and deliver some promising results.

This short paper discusses continually updated causal abstractions as a potential direction of future research. The key idea is to revise the existing level of causal abstraction to a different level of detail that is both consistent with the history of observed data and more effective in solving a given task.

Performance metrics-driven context caching has a profound impact on throughput and response time in distributed context management systems for real-time context queries. This paper proposes a reinforcement learning based approach to adaptively cache context with the objective of minimizing the cost incurred by context management systems in responding to context queries. Our novel algorithms enable context queries and sub-queries to reuse and repurpose cached context in an efficient manner. This approach is distinctive to traditional data caching approaches by three main features. First, we make selective context cache admissions using no prior knowledge of the context, or the context query load. Secondly, we develop and incorporate innovative heuristic models to calculate expected performance of caching an item when making the decisions. Thirdly, our strategy defines a time-aware continuous cache action space. We present two reinforcement learning agents, a value function estimating actor-critic agent and a policy search agent using deep deterministic policy gradient method. The paper also proposes adaptive policies such as eviction and cache memory scaling to complement our objective. Our method is evaluated using a synthetically generated load of context sub-queries and a synthetic data set inspired from real world data and query samples. We further investigate optimal adaptive caching configurations under different settings. This paper presents, compares, and discusses our findings that the proposed selective caching methods reach short- and long-term cost- and performance-efficiency. The paper demonstrates that the proposed methods outperform other modes of context management such as redirector mode, and database mode, and cache all policy by up to 60% in cost efficiency.

We present SODA: the first publicly available, million-scale high-quality social dialogue dataset. Using SODA, we train COSMO: a generalizable conversation agent outperforming previous best-performing agents on both in- and out-of-domain datasets. In contrast to most existing crowdsourced, small-scale dialogue corpora, we distill 1.5M socially-grounded dialogues from a pre-trained language model (InstructGPT; Ouyang et al., 2022). Dialogues are distilled by contextualizing social commonsense knowledge from a knowledge graph (Atomic10x; West et al., 2022). Human evaluation shows that dialogues in SODA are more consistent, specific, and (surprisingly) natural than prior human-authored datasets - e.g., DailyDialog (Li et al., 2017), BlendedSkillTalk (Smith et al., 2020). In addition, extensive evaluations show that COSMO is significantly more natural and consistent on unseen datasets than best-performing dialogue models - e.g., GODEL (Peng et al., 2022), BlenderBot (Roller et al., 2021), DialoGPT (Zhang et al., 2020). Furthermore, it is sometimes even preferred to the original human-written gold responses. We make our data, models, and code public.